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In this paper we extend replica bounds and free energy subadditivity arguments to diluted spin- 
glass models on graphs with arbitrary, non-Poissonian degree distribution. The new difficulties 
f») ' specific of this case are overcome introducing an interpolation procedure that stresses the relation 

, between interpolation methods and the cavity method. As a byproduct we obtain self-averaging 

• identities that generalize the Ghirlanda-Guerra ones to the multi-overlap case. 

I. INTRODUCTION 

The replica and the cavity methods play a fundamental role in the analysis of mean field disordered systems, where 
they suggest a low temperature glassy phase with a great extent of universality [1] . 

With the technique of "replica symmetry breaking" (RSB) physicists have been since a long time able to describe 
in great detail the statistical properties of the configurational space of "fully connected" disordered models, where 
J]3 ■ each degree of freedom interacts with an extensive number of neighbours. 

The last times have seen important progresses in extending the analysis to "diluted models" , where each degree of 
freedom interacts with a finite number of randomly chosen neighbours. Although the equations describing RSB in 
these models were well known already in the past, it is only with the recent introduction of a population dynamics 
algorithm [2,3] that it has been possible to tackle the subtleties of the glassy low temperature phases. Thanks to this 
' algorithm and its specification to zero temperature it has been possible to analyze at the level of "one step replica 
symmetry breaking" (1RSB) a great variety of problems relevant for physics of glassy systems or computer science 
or both. These range from the Viana-Bray model for spin glasses [3], where the 1RSB is expected to give just an 
approximate result, to the diluted p-spin model - or XOR-SAT problem [4] -, the Biroli-Mezard lattice glass model 
■ [5] , models for efficient error correcting codes [6] , the random K-SAT problem and graph coloring [7-9] , where at least 
\Q ' at zero temperature 1RSB is thought to be exact. 

The population dynamic algorithm, originally thought for computing averages over random graph disorder, has 
been suitably generalized to deal with single samples, and in conjunction with decimation algorithms has been proved 
to be effective to solve random K-SAT or graph coloring in essentially linear time up to very close to the SAT/UNSAT 
transition threshold [8] . 

Despite these successes, together with a large wealth of important rigorous results on both mean field and short 
range models [10-12], the theoretical foundations of replica and cavity methods remain unsatisfactory. On one hand 
replica methods rely on a non-unique analytic continuation of the integer moments of the partition function which 
is difficult to control mathematically; on the other, the cavity method is based on physically sound, but unproven 
*^ assumptions on the nature of the low temperature pure states. 

A great deal of mathematical work has been devoted to the analysis of disordered models with rigorous -i.e., 
conventional probabilistic- methods, which in many cases allow to control the RS high temperature case, and in the 
remarkable analysis of Talagrand [11] for the fully connected p-spin for large enough p, even the low temperature one. 
A different approach has been put forward by Guerra [13], who was able to prove that in a large class of long range 
models the free energy can be written as the sum of the replica expression plus a reminder which, by simple inspection, 
is proven to be positive. This was the first proof that the replica/cavity Ansatz is in some cases a variational Ansatz, 
therefore justifying the replica rule that the free energy should be maximized with respect to the replica parameters. 1 
The technique of Guerra is based on the idea of interpolating the original model with a pure paramagnet with suitably 
chosen local external fields. In the original formulation the interpolation was performed tuning the relative strengths 
of the couplings and the external fields, and relayed heavily on the Gaussian and long range nature of the interactions 
among spins. This technique has been employed in several papers dealing with long and short range models [14]. 
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1 Cases are know in replica theory where the right free energy extremum is a saddle point, but this does not happen in the 
mentioned cases. 



1 



In a previous paper [15], hereafter referred to as I, two of us have shown how to generalize the technique of the 
interpolating model to diluted models on graphs with Poissonian degree distribution. In that case, the interpolation 
is done progressively removing addends in the Hamiltonian, and compensating in average by the introduction of 
appropriate external fields on random sites of the lattice. The method was then interpreted as a variational version of 
the cavity method. The result was that the free energy could be written as a sum of a replica/cavity like contribution 
plus a remainder. For the even p diluted p-spin model and the even K K-SAT problems, simple considerations not 
relying on the actual solutions to the replica/cavity equations, allowed to conclude that the remainder was positive. 
This means that the replicas give free energy lower bounds to these models. Specified to zero temperature, this proves 
that replicas give an upper bound for the satisfiability threshold, that is the value of the degree of connectivity below 
which all the terms of the Hamiltonian can be minimized at the same time. The cases of odd p or K led to an 
expression for the reminder whose sign cannot be obviously decided, thus leaving as an open question if the recently 
obtained 1RSB solution for the random 3-SAT problems or the odd p-spin case actually give free energy lower bounds 
and SAT-threshold upper bounds. 

As we said, the results were confined to models on Erdos-Renyi (hyper)-graphs which have Poissonian degree 
distribution. However, the replica and the cavity methods have been successfully applied for spin models on more 
general graphs, where the degree distribution is basically arbitrary. 

Motivated by that consideration, in this paper we generalize the analysis to random graphs with arbitrary connec- 
tivity distribution. Here, differently from the Poissonian case, the compensation in average is not possible and the 
erasure of a term in the Hamiltonian requires detailed compensation through the introduction of proper fields on the 
sites belonging to the erased clause. The new difficulties arising in this case can be overcome employing the property 
of self-averaging of appropriate observables, leading to identities on multi-overlap distributions that generalize the 
Ghirlanda-Guerra identities [16] for the usual overlaps. This new feature is common both to the RS and the RSB 
case. For exposition simplicity we will limit ourselves in this paper to the discussion of the RS case. 

The paper is organized as follows: in section 2 we define the model and introduce the notations, in section 3 we 
introduce the interpolating models, which we use in sections 4 and 5 to find representations of the free energy as 
main terms plus remainders, suitable respectively to prove replica bounds and subadditivity. In section 6 we use the 
self-averaging to estimate some terms in the remainders and generalize the Ghirlanda-Guerra identities. Section 7 is 
devoted to showing the positivity of the remainders, and finally in section 8 we discuss some conclusions. 

II. DEFINITION OF THE MODELS AND SUMMARY OF MATHEMATICAL NOTATIONS 

A. The Models 

In this paper we consider diluted spin models on random graphs with arbitrary degree distribution, consisting in a 
collection of N Ising ±1 spins S = {Si, Sjv}, interacting through Hamiltonians of the kind 

M 

n^(s,3) = J2H JM (s^,:.,s^) (1) 

where the indexes if are i.i.d. quenched random variables chosen in the following way: one first extracts a set of site 
degrees fcj, representing the number of clauses where Si appears, (i = 1, N), as i.i.d. from a distribution pk- Then 
the configuration of indexes if are chosen uniformly among all the possible ways respecting the prescribed degrees. 
In other words, the joint probability of all the indexes {if} will be proportional to 

N M p 

II 5 (EX>r .*-**) ( 2 ) 

i=l fi=l 1=1 

where both <5's appearing in (2) denote the Kronecker symbol. We will concentrate on regular degree distributions 
where all the moments (k l ) = ^2kPkk l are finite. The number of clauses M is therefore itself in principle a random 
variable given by M — ^ ^i, ^ s average (M) = aN will be proportional to N, with a = (k)/p and it will have 

small 0(VN) fluctuations around its average. For this reason, we will often treat M as a constant, making only a 
0(1/N) error in the free energy. As it will become soon clear, the subscript in the clauses indicates dependence 
on a single or on a set of quenched random variables. 

We will treat in a unitary way the case of the the p-spin model [17] or random XOR-SAT problem of computer 
science, and the random K-SAT model. In the p-spin model the clauses have the form 
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Hjm(S^, ...,S^) - J^S^ ■ ... -S^ (3) 

where J M will be taken as i.i.d. random variable with regular symmetric distribution (i(J) = J). Particular 
attention will be given to Viana-Bray model corresponding to p = 2. In random K-SAT model the clauses have the 
form [18] 

ff J( ,,(5 if ,...,5 i? )-n — ^— L > ( 4 ) 

(=1 

where the quenched variables J% = ±1 are i.i.d. with symmetric probability. 2 The number p of spins appearing in a 
clause is usually called K in the K-SAT problem, but for uniformity of notation we will deviate from this convention. 

In I the indexes if of the spins appearing in the clauses were chosen with uniform probability giving rise to 
random graphs with Poissonian degree statistics, and the treatment was based on the peculiar property of the Poisson 
distribution. Here we show that thanks to self-averaging of certain quantities, the validity of the results does not 
depend on the specific form of the graph degree distribution. 

B. Notations 

Let us establish some notations. We will need several kinds of averages: 

• The Boltzmann-Gibbs average for fixed quenched disorder: given an observable A(S) 

= E s ^(S)exp(-/?H(S,J)) 
Z 

where Z — ^ s exp(— f3H(S, J)) and (3 is the inverse temperature. 

Obviously, u)(A), as well as Z will be functions of the quenched variables, the size of the system and the 
temperature. This dependence will be made explicit only when needed. 

• The disorder average: given an observable quantity B dependent on the quenched variables appearing in the 
Hamiltonian, we will denote as E{B) its average. This will include the average with respect to the J variables 
and the choice of the random indexes in the clauses as well as with respect to other quenched variables to be 
introduced later. 

• We will need in several occasions the "replica measure" 

Sl(A 1 ,...,A n )=w(A 1 )...w(A n ). (6) 

• Moreover, throughout the work we will consider modified versions of the original Hamiltonians that will depend 
on a discrete dilution parameter t. Both Boltzmann and disorder averages will depend accordingly on t. The 
original averages will correspond to t = M. 

• Another notation we will have the occasion to use is the one for the overlaps among I spin configurations 
{S*\ SI 1 }, out of a population of n {S}, 5f}: 



N 

(ai,...,a t ) _ 

N 



1 N 

Tr£ST-...-S?' (l<a r <n Vr), (7) 



and in particular 



1 N 

q (n )=q (l,...,n ) = _J2st.....S?. (8) 



i=l 



2 While the assumption of a symmetric distribution //(J) = n{—J) will play an important role in establishing the free energy 
bounds, the precise form of the distribution will not be essential. 



3 



• Finally, we will denote as Tj the set of clause indexes where the i'th spin appears and V M the set of spin indexes 
belonging to the clause /x. 

In the following we will need to consider averages where some of the variables are excluded, e.g., the averages when 
a variable is erased. These averages will be denoted with a subscript —u^, e.g., if an lo average is concerned the 
notation will be u> (•)• Other notations will be defined later in the text whenever needed. 

— u 4 * v ' 

Our interest will be confined to bounds to the free energy density Fn = ~~fTN-E\ogZ and the ground state energy 
density Uqs — lmijv->oo 1/NE [min (i/jv)] valid in the thermodynamic limit, so that 0(1/N) will be often implicitly 
neglected in our calculations. 

III. INTRODUCING THE INTERPOLATING MODELS 

We will use the technique of interpolating models for the purposes of showing the existence of the infinite volume 
free energy, and of proving replica bounds. For exposition reasons, we will explain the method first for the replica 
bounds and later for the existence of the free energy 

A. The replica/cavity bounds: clause deletion versus fields compensation 

In order to prove the free energy bounds, we will use an iterative discrete graph pruning procedure where at each 
time step t decreasing from M to 0, we erase the clause labelled by t and compensate this reduction by the introduction 
of some auxiliary fields u* t on the sites i\ (I = 1, ...,p) belonging to the clause t (see Fig.(l)). At each time step t we 

have a different model with Hamiltonian 

t M p 

wW[S] = ^F J(M) (5 i ,,...,%)- zZ<t Si " (9) 

M=l ju=t+l 1=1 

which interpolates between a simple paramagnet made of non interacting spins at time and the original model 
at time M. The clause removal procedure is very similar to the analogous operation of spin and clause addition 
usually performed in the cavity method, the difference being that here the deletion is explicitly compensated by the 
introduction of external fields. A key point in obtaining the free energy replica/cavity bounds is to assume that the 
external fields are random variables obeying the statistics of the cavity fields in the cavity approach. 

In order to explain this point, and to motivate the introduction of the definitions below, let us remind some formulae 
of the cavity approach. In that context, one singles out the contribution of the clauses and the sites to the free energy 
and defines cavity fields and respectively as the local field acting on the spin i = in absence of the clause fi 
and the local field acting on i due to the presence of the clause /z only. If we define Z[Si] as the partition function of a 
given sample with N spins where all but the spin Si are integrated, and Fm-% as the free energy of the corresponding 
systems where the spin Si and all the clauses which belong to it are removed, in the cavity approach one assumes that 

z[s^e-^U £ e -^^.^.-.^)^EL 2 ^^r 

fiGTi S-fj. ,...,S,/j. 

2 V 

= e -0F N ,- i "Q B <S) e Pu»S K (1Q) 

Notice that in general, if (10) had to represent the exact integration of N — 1 spins in in a finite N systems, effective 
Z-spin interactions (I = 2, ...,p) should also be present among the spins explicitly appearing in the r.h.s. of (10). Given 
the peculiar topology of random graphs, where loops of interacting variables have generically length 0(\og(N)), these 
coupling can be expected to be small and are neglected in the approach. This is the reason why we wrote « instead 
of =. The constant = e _ ^ AF ^ ' is interpreted as a suitable shift in the free energy due to the contribution of 
the clause \i for fixed value of the spin i. We notice that denoting as J, and renaming the fields in (10) into 
hi, ...,h p -i, Eq. (10) defines functions 

uj{hi,...,h p -i) and Bj(hi, h p -i) (11) 

that one can compute explicitly using the form of the clauses of the different models. For instance, in the case of the 
Viana-Bray model, where p = 2, one has: 
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Uj(h) = — tanh 1 [tanh(/?J) tanh(/%)] 



£j(M 



2cosh(/?J) cosh(/3/i) 



cosh(/3u,/(/i)) 

Always in the cavity approach one closes the set of equation imposing the self-consistent equations 



h 



E 



(12) 
(13) 

(14) 



Following the intuition based on Eq. (10) we will define the external fields u% in the interpolating model as verifying 
the relation 



u % — u .jm {9 if i ••• j 9% 1 9% 1 -i " "1 9% ) 



(15) 



where the p — 1 arguments g will be independent variables with suitable statistics. We do not write at this point a 
relation analogous to the self-consistency relation (14). This equation will be obeyed in average when the statistics 
of the fields g is chosen in such a way to optimize the replica bounds. The key point of the procedure, consists in the 
choice of the distribution of the primary fields g^ . Different free energy bounds can be obtained assuming the type 

of statistics implied by the different replica solutions. 





t t-1 

FIG. 1. Clause erasure and balancing fields addition for a particular random clause fj,. The hyper-edge interaction is drawn 
in the factor-graph notation with square nodes representing clauses and circles representing spins. Dotted lines represent the 
added balancing fields. Only clause /i and corresponding variables and fields are explicitly named. 

In this paper we concentrate on the replica symmetric bound, that will be obtained considering the primary fields 
g^ as i.i.d. variables drawn from a distribution G(g). Correspondingly the u's will be distributed according to Q(u) 

verifying 

(16) 



Q{u) = J dgi G(.gi). ..£%_i G(g p ^i)(5(u - uj(gi, 
Writing the free energy of the interpolating model as 



F N (t) = -—E\ogZ(t), 



(17) 
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we observe that the average free energy Fjf = Fn(M) of the original model can be written as 

M 

F N = Y,&N(t) + F{0) (18) 
t=i 

where we defined the discrete time derivative of the free energy as 

A N {t)=F N (t)-F N (t-l). (19) 

The bound will consist in comparing Ajv(i) with the corresponding replica symmetric expression. 

The extension to 1RSB bounds, although technically more involved, does not present additional conceptual difficul- 
ties. In I, it has been shown how to provide 1RSB bounds in the Poissonian case. One needs to assume that the fields 
are still independent variables, but subject to site and clause distributions which are themselves random and 

independent from one another and subject to the common functional distribution Q{G). The appropriate definition 
of interpolating free energy will depend on a parameter m £ [0, 1]: 

F »® = 1 7 ^E 1 \ogE 2 Z(tr. (20) 

We denoted by E 2 the average over the fields for fixed distributions G% and by E\ the average over the distributions 

l i _ l i 

G% as well as over all the other quenched variables. For t — M the E 2 average is immaterial and formula (18) holds. 
For general degree distribution one could follow the same procedure, but in order to keep this paper within a reasonable 
length we will not present here this case. 



B. The thermodynamic limit: a cut and paste procedure. 

The aim of this section is to interpolate between the original system of N interacting spins and two separate models, 
respectively with N\ and N 2 spins {N\ + N 2 = N) and to show that the free energy is subadditive [19]. For notational 
simplicity we consider here explicitly just the case of the Viana-Bray model, with Hamiltonian H = — 5Zu=i J^S^ . 
Inspired by the construction of the previous section we start from the model with N spins and consider the sets of 
the first Ni spins and the one of the remaining N 2 spins. Each of the M clauses, will either belong to the first 
sub-system, if i^i^ G {l,...,Ni}, or to the second if ii,i 2 £ {Ni + 1, ...,N} or they will be "bridge" clauses if one 
of the indexes is less or equal than Ni and the other is greater than Ni. Let us denote as Mi(0), M 2 (0) and M&(0) 
the number of clauses of the different types, respectively. We define our interpolating model via an iterative "cut and 
paste" procedure where at each time step we select at random two bridge clauses, we cut them, and we reconnect the 
spins belonging the first sub-system between themselves with a new random coupling, and similarly for the two spins 
belonging to the second sub-system (see Fig. (2)). 




FIG. 2. Cut and paste procedure in the Viana-Bray model. The groups of constraints not belonging to Mb and of graph 
sites not participating to the constraints in Mt are generically represented as dashed lines sets. 
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In such a way at each time step t = 1, ...,Mb(0)/2 (we suppose for simplicity to choose the ordering of the spins 
in such a way that Mj,(0) is even) each spins conserves its original connectivity and the number of clauses of the 
different kinds are modified as Mx{t) = Mi(i - 1) + 1, M 2 (i) = M 2 (t - 1) + 1 and M b (t) = M b (t - 1) - 2. At the 
end of the procedure we have two separate models with, respectively, Ni spins and Mi(0) + Mft(0)/2 clauses and iV 2 
spins and M 2 (0) + Mft(0)/2 clauses. Notice that Mi, M 2 and Mb are random variables with average proportional to N 
and small 0(y/~N) fluctuations. As in the case of M, we will neglect these harmless fluctuations in our analysis. It is 
easy to realize that the graphs associated to the resulting non-interacting models are chosen with uniform probability 
among the ones having the correct connectivities for each spin. This can be explicitly checked by an elementary 
induction calculation. Each step of the cut and paste procedure transforms the uniform distribution on the graphs 
with prescribed connectivities ki of the sites and number of clauses of the three types Mi(i), M 2 (i) and Mb(t), into 
the uniform distribution with the same connectivities but with clause numbers Mi(t) + 1, M 2 (t) + 1 and M h (t) — 2. 

Therefore, in analogy with (18) we have 

F N = 1 ±F Nl + -?-F N2 - J2 A WW, (21) 

t=i 

where A' N (t) is the discrete time derivative along the cut and paste procedure. As will be shown in section 5, A' N (t) 
is non-negative for the models we are considering, as long as p is even, thus implying subadditivity of the free energy 
and therefore the existence of its infinite volume limit. 



IV. THE RS BOUND 



Let us start with a few observations independent of the chosen statistics of the primary fields, and therefore valid 
both for the RS and for the RSB estimate. 

In order to compare the free-energies F(t) and F(t — 1), let us consider Z(t — 1) and isolating the terms containing 
the fields u% observe that this can be written as 



Z(t - 1) = Z_ ut (t - l)Jl-J ] (e ^ 'I •« 



(22) 



where Z_ u t(t — 1) and uj^ u }\-) are respectively the partition function and the Boltzmann averages corresponding to 
the Hamiltonian at time t — 1 in absence of the fields u* t . In the same way, isolating the t-th clause term in Z(t) and 

noticing that Z_j(t) (t) = Z_ u t(t — 1) and (•) = uj^ u }\-) we can write 



Z(t) = Z_ ut (t lV^Ce"^^'-'^ 5 ). 
Using these relation in the definition of the RS free energy time derivative Ajv(t) we find: 



NA N (t) = -T 



E (] ntl ;^ 1) (e-^ w(S 'i , -^ ) )) - E { In ^ >(e^ ^ *<* ) 



(23) 



(24) 



Now we compare this term with the one obtained in the cavity approach (supposing that the statistics of the cavity 
fields h coincide with the one of our external primary fields g): 



(t-l), -fSH (t) (S it ,...,S it ) 



nf=i2cosh(/3/ l *) 



E 



-0H j(t) (s iV ...,s itp )+0j2 P l= i ht i* s > 



and in the same way 



This leads to the expression: 



(25) 



(26) 
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NA N (t) « —T 



Slog 2~ p ^ e -PH jW (s lt ...,s p ) jj(i + ^tanh(/3/i* t )) 



S\....,S P 



V 

~ E H (log(cosh(/3u* ? )) + log(l + tanh(/3/i**) tanh(/3^))) 



;=i 



(27) 



The replica symmetric approximation to this expression consists in assuming that the statistics of the cavity fields 
/i't coincide with the one of the external fields <?' t , and we call A*(i) the expression corresponding to (27), once these 
substitutions have been made. 

Now, wc go back to our approach where, it is important to emphasize, we are not assuming the validity of approxi- 
mations like (25, 26). In order to get a control of the free energy we add and subtract A*(t) from the expression (24) 
of Ajy(t). It is also useful to add and subtract the term 



tJ^E log(l + J'-P ) tanh(/?u* f )) 



(28) 



Rearranging terms in Ajy and taking into account that 



1 N 

F{0) = -^£ lo g(2cosh(/3 £ <)) 



i=i 



we rewrite the free energy as: 

F N = F var [G] + 1 jr R[G, t] + ±J2 ^ G > *] + °( W 



M 



A/ 



t=l 



t=l 



Here, we have isolated the "variational term" S„ or [G] 



1 N 

F var \G] = -_s]Tlog(2cosh(/? ^ <)) 



i=l 
M 



0N ^ 



Slog 2~ p ^ e -PH jW (Su...,s p ) jj(i + ^tanh(/? fl *«)) 



Si,...,S p 



1=1 



v 

-EJ2 (log(cosh(/3u* f )) + log(l + tanh(/3 5 **) tanh(/?uj f ))) 
i=i 



from the remainders 
J2[G,i] = -T 



£ (log^Ce-^^i--^))) - £ K>g(l + ^-; ) (^)tanh(/34) ) 



(29) 



(30) 



(31) 



-Slog [ 2~" e-^w( Sl -- S p) JJ(l + Sitanh09 5 *j)) ] + S^T (log(l + tanh(^) tanh(/3t^)) 

Si ... . .Sp /— 1 



(32) 



and 



J2[G,i] = T 



E (logJ^^Ca + ^tanh^))) j -S ( ^ log((l + ^(5^) tanh(/3 U *,)) 



/=i 



(33) 



Expressions (31,32,33) are suggestive on how to use the interpolating model method to analyze single samples [20]. 
Moreover, we can simplify them observing that 



• All the u and g fields appearing are statistically independent and one can therefore assign them arbitrary indexes. 
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• The marginal of the index probability (2) with respect to the indexes appearing in the i-th clause is the uniform 
distribution on {1, ...,N} P . The indexes appearing in the averages in (31,32,33) are therefore immaterial. 



We finally find: 



PF var [G] = -£log(2cosh(/?^ U; )) 

i=i 

ElogU-P e-0 Hj &<~- s »)f[(l + $tanh(/3 5; )) 

\ Su-,S P 1=1 

-E J2 (log(cosh(/9«0) + log(l + tanh(/3 ff; ) tanh(/3uj))) 



i=i 



R[G, t] = -T 



E (logJXPie-^^-^)) -pE (k>g(l + ^ ) (^ i )tanh(/3 U )) 



(34) 



-Elog\2-P e~P Hj{Sl '-' S v ] jj(l + Sitanh(/9fli)) ] + (log(l + tanh(/3g) tanh(/3u))) 

Si,...,S p l—l 



(35) 



R[G, t]=T 



E (logwi^^JJCl + $f tanh(/3u,))) ) - p£ (k>g((l + w^to*) tanh(/?u))) 



z=i 



s/to. 



(36) 



Direct inspection upon optimizing over the G function shows that F var [G] coincides with the free energy found with 
the replica/cavity method in the replica symmetric approximation. The first remainder term R[G,t] is analogous to 
the one one finds on Poissonian degree graphs and will be dealt with in section 7. Conversely, R was absent in the 
Poissonian case thanks to the possibility of compensating in average the clause removal procedure and represents a 
new difficulty for graphs of arbitrary connectivity. At a first sight it would seem difficult to say anything in general 
about its behaviour. However, we will see in the next section that, thanks to the self-averaging property of extensive 
quantities, this term becomes vanishingly small in the thermodynamic limit. This is the only point, besides trivial 
0(1/N) terms neglection where the large N limit enters in our estimates. 



V. THERMODYNAMIC LIMIT 



In order to analyze the cut and paste algorithm let us introduce Z(M\, M 2 , Mb) as the partition function of the 
interpolating model when the number of various kind of clauses are given by M\,M 2 ,Mb, and u>tM 1 ,M 2 ,M b )(') as the 
corresponding Gibbs average. A step of the algorithm, at generic time t, sends Z{M\,M 2 ,Mb) to Z{M\ + 1,M 2 + 
1, Mb — 2). Let us consider s consecutive steps. If we denote as i\,V 2 and j{,j 2 , r = 1, . . . , s the indexes of the bridge 
clauses we cut and as J} and J 2 the corresponding couplings, we can write: 

Z(M 1 ,M 2 ,M b ) = Z(M 1 ,M 2 ,M b - 2 S )^ (Ml , M2iMb _ 2s) (e^- (J ' S ^ S *2 +J '^T s ^)) 
Z{M 1 +s,M 2 + s,M b - 2s) = Z{M x ,M 2 ,Mb - 2 S )^ (Ml , M2iMb _ 2s) (e ,3 ^- (J ' S *I S ^ +J ' Ss 2 S ^ ) ). (37) 

(s) 

The total average free energy change A y N ' after the s steps is given therefore by: 

t+s-1 / 

/?JVA# =0N ]T A' N = El log^ (Ml ,M 2 ,M b - 2s )(n( 1 + tanh(/3J r 1 )^^)(l + tanh(/3J 2 )S jT %)) 

T=t \ r 

- log u {Ml< M 2 , Mb -2 S ) (11(1 + tanh(/3J r 1 )^5' jT )(l + tanh^J, 2 )^^))^ . (38) 
Similarly to section 4 we add and subtract suitable terms to A$ , to write: 
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0NA$ = 2sE {logLu {MlM2 . Mb _ 2s) (l + tanh(/3J 1 )S 4l S J2 )} 

-sE {logo; ( M 1 ,M 2 ,M i) -2 S )(l + tanh^J^S^Sj-J} - sE {logw (Ml ,M 2 ,M t -2«)(l + tanh(/3J 2 )S i2 S j2 )} + T (39) 

where, for symmetry reasons, we have omitted the index r. Since we have factored the Gibbs averages, the remainder 
T will contain terms of the type 



E 



log W(MljM2 , Mb _ 2s) ([](l + tanh(/3J r 1 )5^S*, T )(l + tanh^J 2 )^^)) 

T 

(1 + tanh^J^SirSj-r) - ^ logu; (Ml; M 2 ,M b -2 S )(l + tanh(/3J 2 )^S i; 



(40) 



As we will briefly discuss in section 6, the same kind of self-averaging properties that will ensure the vanishing of 
R in the thermodynamic limit, will also guarantee that T — ■> 0. At this point, as in I, one expands the logarithms 
in absolutely convergent Taylor series of the variables tanh(- • •). Recalling the fact that the distribution of the J 
variables is even, one obtains 



0NA[ s) 



n=l 



(tanh'" (3 J) 
2n 



E [(u(M 1 ,M2,M b -2s)(Si 1 Sj 1 )) 2n - 2(u( Ml . M2tMb _2s){Si 1 Si 2 )) 2n + {u(M 1 ,M 2 ,M b -2s){Si 2 Sj 2 )) 



2-f/l 



Define X 1; I 2 as the set of 2s indices in the first and second sub-system, respectively, corresponding to the deleted 
bridge clauses. Then it is easy to realize that, conditionally only on all the non-deleted clauses, the random indices 
ii,ji and i 2 , 32 are independently and uniformly distributed on T\ and X 2 , respectively, apart from an error term of 
order 0(l/s). This error term arises because of the constraints i\ ^= i 2 , ji =/= ,]2, which becomes weak for s large, since 
the number of non-diagonal configurations is 0(s 2 ), while that of the diagonal ones is O(s). Introducing, in analogy 

with section 2, £l(M 1 ,M 2 ,M b -2s) as the replicated version of the measure W(M 1 ,M 2 ,M b -2s) an d <7i 2 ™\ as 



9i 



ieii 



?2n 



?2n 



(42) 
(43) 



iei 2 



A N ' can be rewritten as 



0NA<$ 



n=l 



(tanh 2 " (13 J)} 
2n 



En 



(M 1 ,M 2 ,M b -2s) 



( ~(2n)_-(2.) )2 l +0(1) _ 



(44) 



The first term is clearly positive. As for the term O(l), it becomes negligible in the limit of large s, since the sum in 
(21) contains 0(N/s) terms of this kind and has a pre-factor 1/N. 

Notice that this expression is different from the one found in the procedures used in the Sherrington-Kirkpatrick [19] 
and in the Poissonian [15] cases where, in the analogous expansions, one finds polynomials in the usual multi-overlaps 
<2i 2 "\<72 2? ^ defined in section 2, depending explicitly on the ratio N±/N. 



VI. CONSEQUENCES OF A SELF-AVERAGING PROPERTY. 

Formulae (34,35,36) provide an exact representation of the free energy. We will see in the next section that at 
least for even p, the remainder i?[G, t] is non negative for all distributions G for which it makes sense. This is not 
enough to prove that F is limited from above by the replica free energy; a control of the term R[G, t] is also required. 
As we show below, this control is guaranteed by the self-averaging property of suitable extensive quantities, which 
follows from general thermodynamical convexity arguments, first employed in the context of mean field spin glasses 
by Guerra in [21]. 

As a preliminary fact, notice that, as it is clear from (30), in order to establish the lower bound for the free energy 
it is sufficient to show that R[G, t] vanishes for t < M — eN, for arbitrary e > which will be let tend to zero in the 
end. 
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Let us consider the interpolating model at time t — 1, and rewrite its Hamiltonian (9) as 

M p 
H=t 1 = 1 

where Ho contains the clauses indexed 1, . . . , t — 1. Let us now consider the quantities 

M p 

V 



(45) 



-, M p 



(46) 



M=* (=1 



which can be expected to be self-averaging with respect to the Boltzmann and the quenched averages, for any integer 
k. Indeed, one has that generically 



L(k,l)= lim L N (k,l) = lim E(u>((f> k <pi) - w{<j) k )uj{4>i)) = 0, 



(47) 



where what we mean by "generically" will be clarified below. For the moment, let us explore the consequences of this. 
Notice that we can write 



M p 



LN{k,i) = w J2 E E 



■ logu_ 



(e *r T s ) 



fj,,is=t r,s— 1 

Thanks to the average E the sum is immaterial and we can write that 



(48) 



(M - tf 
N 2 



E 



d d 



f3{u» S^+u^S^) 

! ' S 



(49) 



where fj, ^ v or r ^ s. Notice that the pre- factor is of order 1. Next we notice that, conditionally only on the clauses 
1, . . . , t — 1 which have not been removed, the random variables Of, i v s are independent and identically distributed as 



Pffi = i) oc (ki - ki(t)), 



(50) 



where fc, is the degree of the site i in the original system, and ki(t) is its degree at time t—1. Of course, ki — ki(t) > 
is just the number of deleted clauses which involved the i'th spin. In particular, choosing [i = v = t, r = 1, s = 2, one 
can write 



E 



, t ski t w 9 d , , P{u\ t S it +u\ t S it ) 



0. 



(51) 



Observing that this identity has to be valid for all k and I, we find that with probability 1 with respect to the 
distribution of ii* t and u l . t , 



d d „ /, , £(u* s it +u* s it ) 

-£_„._„< logw_ B . t (e ii 22 



0. 



(52) 



This implies that 



_ u * t ( logw_ u * t _ u * t (e 



0(u% S it +u* S it ) 

1 1 '2 2 ) 



£- u * t -„* t logw_„ tt _ u t t (e i i) 
+E_ u t _ u t log w_„* _ u t (e 22) 



(53) 



and that 

E_ u t_ u t logw_ u t (e i !) 



)=E_ u t t _ u t I logo;_„t t _„t t (e i 1 2 2 ) 1 _ E_ u t t _„* I log w_ u * t _„* t (e 2 2) 

/ *1 *2 V *1 l 2 / 'l '2 \ l l *2 / 



logw_ 



(e '1 



(54) 
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where the equivalence sign ss means in the previous formulae and in the rest of the section that both quantities on the 
two sides of an equation tend to the same value in the thermodynamic limit. It is easy to realize that these identities 
can be generalized to ones involving arbitrary number j of fields: 

l i "j V *i 'j / r=1 l i "j V / 

If we now take j = p and recall (33), we conclude that self-averaging of the (f>k implies that R[G, t] tends to zero in 
the thermodynamic limit. 

Similarly, to prove that the remainder T of section 5 vanishes, one has to consider a Hamiltonian of the form 

M 

H = Ho — ^ w^S^S^ 

and to exploit self-averaging of the quantities 

Repeating the steps which led from (47) to (55), one finally finds that T vanishes for N — > oo. 

We are now left with the task of showing that relations like (47) generically hold. The strategy we use has been 
already employed in [21], and more recently in [22]. The idea is as follows: while one is not able in general to prove 
that Eqs. (47) hold for the original model (45), one can perturb it by adding a term NJ2T=i ^k4>k to the Hamiltonian 
TL 1 Afc being real numbers decreasing sufficiently fast with k, so that the infinite volume free energy remains bounded. 3 
In the end, we will be interested in the case where all the A's vanish. Then, as in [21], convexity of the free energy 
with respect to parameters implies that, for almost every choice for their values, the second derivative d\ k FM is 
finite, also in the thermodynamic limit. Since this derivative can be written as 

d\ k F N = NE (^{cg) - luM 2 ) , 

it follows that equations (47) hold, for almost every choice of the A's, if the Gibbs averages are understood to 
correspond to the modified model. In order to conclude the argument and show that R — > 0, we have to remark two 
important points. First, Eqs. (48) will be modified by a harmless remainder of order 0(A), which will disappear in 
the end since we can take the Afe arbitrarily small. Next, the remainder term R of section 4 remains positive, so that 
Eq. (30) gives 

F N > F var [G] + 0(1/N) + 0(A), (56) 

where both the replica-symmetric and the true free energy are those corresponding to the modified system. Finally, 
one employs the fact that the free energy is always a continuous function, as a consequence of its convexity, to deduce 
that (56) holds also for the original system with all A's set to 0. It is important to emphasize that we have not used 
anywhere assumptions of continuity of Gibbs averages with respect to the parameters A, which we cannot prove in 
general, but just simple positivity properties and the continuity of the free energy, which holds as a general fact. 

1. Generalization to multi- overlaps of the replica equivalence identities 

It is interesting to investigate the consequences of Eq. (49) on the distribution of multi-overlaps. To that scope, one 
can perform an expansion in powers of tanh(/3uj) and tanh(/3«j) in (49). Omitting tedious but conceptually simple 
calculations one finds that in the thermodynamic limit, for all r and s, 

min\2r.2s] , n . 

^ (2r- l)l(2a-l)lll {{q q H) ' 1 ' 



3 Results similar to those described below could be obtained considering perturbations of the kind N 1 a X]fcLi ^k4>k with 
< a < 1/2, which even for finite A's modify the free energy or equations (48) only by 0(N~ a ) terms. 
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where the q( 2r ^ and q( 2s ^ are multi-overlaps involving respectively 2r and 2s replicas, and with the notation (q^ 2 ^ -q^ 2s ^)i 
we mean that among the two groups of replica I are in common. Notice that the relations (57) generalize to the case 
of multi-overlaps the Ghirlanda-Guerra identity [16] 

-^(<Zl2<Z34> + 2(q 12 q 23 ) - ^(q 2 12 ) = (58) 

and reduce to it for r = s = 1. More general relations can be obtained from (55) or, in analogy with [16], considering 
considering self-averaging properties of multi-spin perturbations in the Hamiltonian. While we will not pursue this 
route in this paper in full generality, it is clear that p-spin perturbations to the Hamiltonian will give rise to equations 
similar to (57) involving multi-overlaps raised to the p-th power. 

We notice that -as it happens in the Ghirlanda-Guerra case- within the replica method, identities (57) can be 
derived from the requirement of "replica equivalence". Within replica method one introduces multi-overlap order 
parameters Q ai ,...,a r , and self-consistently finds that the multi-overlap averages are given by 

(<Z (r) ) = lim- — — V Q 01 ,...,„ P , (59) 

n^o n(n - l)...(n - r + 1) ^ 



Ol, 



where all the indexes in ^ are different. Similarly, the averages ((q^ ■ q^)i) are given by replica sums of 
Qai,...,a r Qbi,...,b s with / a-indexes coinciding with I 6-indexes, normalized to the number of terms in the sum. For 
instance, ((q^ ■ q^)i) = n ( Tt _i)( ra _2) J2' a b c QabQbc- Replica equivalence states that for all values of ai the sum 



1,71 



a2 ,...,a 

takes the same value. As a consequence, for n — > 

n 



_^ ,l,n 

- J2 Q«u..,* r E Qbi,...,b. = 0{n) -» 0. (61) 

•,ov fei,...,6 s 

Singling out in the sum terms with coinciding indexes in the overlaps and making use of (59), after some algebra one 
finds Eq. (57). Eq. (49) can be seen as the generating function of these identities. 

VII. THE REMAINDER R[G,T}. 

While the control of the remainder R was obtained, in the previous section, without making reference to a specific 
form for the Hamiltonian (1), in the present section we restrict ourselves for simplicity to the p-spin case, where 
Hj(Si 1 , ...,Si p ) = J Si ± ■ ... ■ Si p . Substituting in Eq. (30), using the independence among the external fields and the 
relation (16) one finds for the first remainder term: 



R p - spm [G,t] =~ 



p-i 

E (log(l + tanh^JVi*" 1 ^^...^))) -pEl log(l + tanh(/3J) J] tanh^V^ 1 ' 

J \ i=i 



p 




(p - 1)E ^ (log(l + tanh(/3J) rjtanh(/3 ff; ))J J 

Through this expression we can establish that the remainder is positive for even p. 

Now, we expand the logarithm of the three terms into (absolutely convergent) series of tanh(/3J), and notice that 
thanks to the parity of the J and the g distributions, they will just involve negative terms. We can then take the 
expected value of each term and write 



1 ' J (3 ^ 2n 

n=l 



2" „ , 

(u/*- 1 ^ . . . S ltp )) 2n - p(tanh 2 " /3ff)S- 1 (^ ) (S ii )) 2n + (p - l)(tanh 2 " fig)* 
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In analogy with section 2, we introduce f2(* _1 ) as the replicated version of the measure a/* -1 ). Next we notice that, 
as in section 6, conditionally only on the clauses 1, . . . , t — 1 which have not been removed, the random variable i\ is 
distributed as 



P{i\ = i) oc fa - h(t)). 



Then, defining 



and 



Z(t-l) 



„(2n) 



it is easy to realize that 



Z_ u i(t-1) 



p{M-t + l) 



(64) 



(65) 



(66) 



n=l 



{r {2n) f - pr (2n) (tanh 2n Pgf" 1 + {p - l)(tanh 2n (3g) p q +0(1/N). 



(67) 



The observable ^ can be seen as the operator which annihilates one of the external fields Ui from site i. Indeed, for 
any observable A one has 



Lj(Atpi)=uj_ u i(A). 



(68) 



The harmless error term 0(l/N) arises because, in order to reconstruct [A 2n ^) p , we added "diagonal" terms where 
at least two indexes i\ and i\, are equal. Since M — t > eN with e > 0, as in section 6, these terms give altogether 
a vanishing contribution in the infinite volume limit. It is interesting to notice that, as found in I, for models with 
Poissonian connectivity degree the expansion of R[G, t] can be expressed in terms of the usual multi-overlap q^ 2n ^ 
defined in section 2. 

In the case of the -ftT-SAT, using definition (4) for the clause Hj, we find relation: 



uj{9i,--,9p-i) = uj({Ji},{gi}) = ^ tann 1 



j-jp-1 ^ l + Jjtanh(/3 g; ) ^ 



i+lnr=i( 



l+Jj tanh(/3g, 
2 



(69) 



where £, = e @ — 1 < 0. Therefore, one has 



R K - SAT [G,t] = ~E 

P 



I / p 
log H (e-P - l)w(JJ 

\ V 1=1 



-p ( log 1 + £w 



1 + J$* |j 1 + J t tanh(/? gi ) ' 



+ 



{9l},J,{Jl} 



( P -i)(.o e (i +f n 1+J 'T" (i% " )), 



(70) 



Expanding in series the logarithms, exploiting the symmetry of the probability distribution functions and taking the 
expectation of each term of the absolutely convergent series we obtain: 



R K - SAT [GA = - fl Y. 



her 



(3 ^ n 

f n>l 



n 



(1 + R n y - p(l + R n )((l + Jtanh^))")^" 1 + (p - !)((! + Jtanh(/3 ff ))")^ 



(71) 
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where we have defined £* = £/(2?) < and R n = E™=i( J '> J .< 0i r ai - a '. 

We notice that both in the p-spin and in the K-SAT problem the remainder can be written as a series of the type 

oo 

R[G,t] = Y,C n ESl[f p {X n ,Y n )\ , (72) 

71=1 

where 

f p (x, y)=x p - pxyP- 1 + (p - l)yP , (73) 

C n > for any temperature, X n 's are suitable combinations of overlaps and Y n : s averages of g-fields hyperbolic 
tangents moments that correctly calculate the overlaps in the corresponding replica symmetric approximation. More 
specifically, for the p-spin 

_ (tanh 2 "(/3J)) 
X n = r< 2 ") 

Y n = (tanh 2 " (/?,?)) (74) 



while for the K-SAT 



(-D" 



0n 

X n = 1 + R n 

Y n = ((1 + Jtanh(/3g))"). (75) 

As noticed in I, for even p the function f p (x,y) is positive for all real x and y thus ensuring the positivity of the 
remainder. For odd p this is not the case as f p (x,y) at fixed y becomes negative for x negative and large enough. 
Although physically one expects that for odd p negative values of X n appear with probability exponentially small in 
N, we have not been able to prove this property in full generality. 

The odd p case is however interesting. In particular, one would like to be able to control the remainder in the case 
of the 3-SAT problem, recently solved within 1RSB scheme [8]. When explicit solutions of the replica equations exist, 
one can try to plug them into the expression of the remainder, to check positivity. Results in this directions, in the 
RS case at zero temperature for K-SAT and p-spin models, will be reported in Ref. [23] . 



VIII. SUMMARY AND CONCLUSIONS 



In this paper we have shown that the free energy of diluted spin glass models with arbitrary random connectivity 
can be written as the sum of a term identical to the ones got in the cavity/replica plus an error term. The expression 
has been obtained through the introduction of an auxiliary model interpolating between the original model and a 
pure paramagnct. The interpolation can be though of as a discrete time dynamical process in which the terms of 
the Hamiltonian are progressively removed, while the removal effect is compensated by the introduction of some 
external fields. The procedure generalizes the previous work [15] on Poissonian graphs, where the compensation could 
be performed in average: at each step one adds there a random number of fields on random sites. In the present 
case, on the other hand, a detailed compensation where one puts a field on each site involved in the erased clause is 
necessary. As a consequence, a new term in the remainder appears. We have shown that, thanks to self-averaging of 
suitable extensive quantities, this new term gives a vanishing contribution in the thermodynamic limit. The rest of 
the remainder is manifestly positive for even p. 

It is also possible to show, as it was done in [24] for models with Poissonian random connectivity, that the free 
energy and the ground state energy are self-averaging quantities, and one can obtain upper bounds, exponentially 
small in the system size, for the probability of large fluctuations. 
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